UbuntuWorld 1.0 LTS - A Platform for Automated Problem Solving & Troubleshooting in the Ubuntu OS

In this paper we present UbuntuWorld 1.0 LTS a platform for developing automated technical support agents in the Ubuntu operating system. Specifically, we propose to use the Bash terminal as a simulator of the Ubuntu environment for a learning-based agent, and demonstrate the usefulness of adopting reinforcement learning (RL) techniques for basic problem solving and troubleshooting in this environment. We provide a plug-and-play interface to the simulator as a python package where different types of agents can be plugged in and evaluated, and provide pathways for integrating data from online support forums like Ask Ubuntu into an automated agent’s learning process. Finally, we show that the use of this data significantly improves the agent’s learning efficiency. We believe that this platform can be adopted as a real-world test bed for research on automated technical support. Building effective conversational agents has long been the holy grail of Artificial Intelligence (Turing 1950). Research in this direction has, however, largely recognized that different modes of conversation require widely different capabilities from an automated agent, depending on the particular context of the interaction; the focus has thus been on approaches targeted at specific applications. For example, conversational agents in the form of chat bots are required to be more creative, responsive and human-like; while for automation in the context of customer service, qualities like precision and brevity are more relevant. Indeed, human agents while providing customer support make a conscious effort to be as structured as possible in their interactions with the user. For successful automation in this particular mode of dialog (that we refer to as end-to-end goal-directed conversational systems or e2eGCS) we identify the following typical characteristics End-to-end. This is the ability of the agent to build and operate on knowledge directly from raw inputs as available from the world, and to generate the desired behavior. Goal-directed. The interactions in these settings are targeted at achieving specific goals, i.e. to solve a particular problem or reach a desired state. General purpose. It is infeasible to build fundamentally different support agents for every possible environment, Copyright c © 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. and hence there must be a learning component to the agent that facilitates automated building of domain knowledge. Adaptive. An agent must learn to adapt to its experience and update its knowledge, and this further underlines the importance of an agent’s capability to learn. Integrated. Finally, the agent must be able to interact with the human in the loop and integrate (and subsequently learn from) human intelligence in order to solve a wide variety of problems effectively. One of the canonical examples of such systems is technical support. As in the case of customer service in general, automation for technical support requires an agent ascribing to the e2eGCS paradigm to be able to: • learn a model or understanding of its environment automatically by means of experience, data and exploration; • evaluate its knowledge given a context, and learn to sense for more information to solve a given problem; and • interact with the customer, maybe in multiple turns, in a natural fashion to solve a given problem effectively. In this paper we specifically address the learning problem, and make a first attempt to lay a pathway towards achieving fully fleshed-out e2eGCS of the future. Technical support is a particular instance of customer service that deals with problems related to the operation of a specific piece of technology, which means there often exists an underlying (albeit unspecified) model to the operation of such a system, and the model learning proposition becomes especially attractive in this context. However, the critical problem here is that the engineers who build the technology, the people who use it, and the ones who provide support for it are often distinct from each other. One solution then would be to make the architects of the system also build the support engine following the same software specifications; this quickly becomes intractable (and might well require its own support!). A more worthwhile alternative is to learn this model automatically. Such an approach, while being considerably simpler to follow, is also likely to be more effective in capturing domain knowledge and providing directed personalized support. The specific domain we look at in this work is technical support in the Ubuntu operating system. This is undoubtedly a real-world environment where support is extremely sought after. Indeed there is a thriving community on the ar X iv :1 60 9. 08 52 4v 2 [ cs .A I] 1 2 A ug 2 01 7 Figure 1: Graph (http://bit.ly/2blmZk1) showing the number of zombie posts from 01/2011 to 07/2016. These are the posts that have remained unanswered for more than 72 hours, and their number is growing exponentially. online Ubuntu help forum Ask Ubuntu, a question and answer site for Ubuntu users and developers hosted on the Stack Exchange network of Q&A sites. Ask Ubuntu currently boasts more than 370k registered users and 238k questions asked till date, and ranks third overall in the family of 158 Stack Exchange communities in terms of traffic or number of users (as of August 2016). A closer look however reveals that this rank is not an indicator of the quality of support. In terms of percentage of questions actually answered (Stack Exchange 2016), Ask Ubuntu operates at a lowly rate of 65%, ranking just five places off the bottom of the list. Further, as shown in Figure 1, the number of posts that go unanswered is exploding in recent times (Ask Ubuntu 2014). While there are many causes that may have led to these dire circumstances, some of which we discuss below, one thing is quite certain Ubuntu needs support, and there isn’t enough of it out there. Motivation Ask Ubuntu’s afflictions may be largely attributed to the following main causes 1. New users clogging up the system with simple problems that experienced users do not care to respond to. 2. Duplicate questions, due to large numbers of users who do not bother to look up existing solutions before posting. 3. An unhealthy newcomer to expert ratio in the community as a result of Ubuntu’s rapidly growing popularity. 4. The continuous roll out of new software/OS versions and corresponding problems with dependencies. 5. Incompletely specified problems, including insufficient state information and error logs leaving members of the community little to work with. We claim here that a large number of these problems can readily be solved through automation. While it may not be reasonable to expect an automated agent to learn the most nuanced details of the Ubuntu OS and solve niche issues that the experts on Ask Ubuntu are more capable of addressing, the large majority of problems faced by users on the forum are readily addressable. These are either (1) simple problems faced by newbies that may be directly solved from Figure 2: Use case querying Ask Ubuntu for guidance. the documentation, whose solutions can be learned from exploration in the terminal; or (2) duplicates of existing issues which may have already been solved, whose solutions may be retrieved using relevant data from Ask Ubuntu. The learning approach then also indirectly addresses issues (3) by freeing up (and in turn tapping into) support from Ask Ubuntu; and (4, 5) since the domain knowledge built up over time as well as local state information sensed by the integrated support agent may be useful in providing more directed and personalized support. Figure 2 provides an illustration of the kind of tasks we are interested in. Consider the simple task of opening a text file. It can be achieved in a single step using gedit, or it can be arbitrarily harder depending on the actual state of the system the agent might need to install gedit if it is not available, and it may need to access the internet and gain sudo permissions to install gedit. This is represented in the top part of the figure. We want our agent to learn these workflows and dependencies on its own, by exploring the Bash environment. Thus when an error comes up, for e.g. regarding administrative rights, the agent knows it needs to execute the sudo command. Of course, this is one of a very large number of traces that the agent will need to explore Figure 3: A schematic representation of the UbuntuWorld 1.0 LTS prototype. before it converges on the correct choices, and this is hard for a setting such as Ubuntu due to the large number of actions that an agent can possibly perform at any given state. Perhaps ironically, we turn to Ask Ubuntu itself in order to make the learning agent’s life a little easier in this regard. As we noted before, users on Ask Ubuntu have been solving these problems for a long time, and their solutions can in fact be used to provide valuable guidance to a learning agent, as shown in in Figure 2 here, the terminal has produced a “Permission denied” error in response to a call to an apt-get command without a sudo prefix. The agent in response queries Ask Ubuntu with the text of the error by using a pre-built Lucene (Lucene 2016) reverse index, and finds answers such as “prefix the command with sudo” that can be used to inform its own deliberative process. The recommended action – in this case, sudo – is then extracted by (TF-IDF based) similarity matching between the top retrieved answers and the description of the actions (we used linux man pages for this) in our domain. Thus, in addition to wanting our learning agent to explore and build a model of Ubuntu, we also want to make sure that the exploration is smart given the abundance of data already available on troubleshooting in Ubuntu. We refer to this as data-driven reinforcement learning, and we expand on this idea through the rest of the paper. Related Work Bringing AI techniques particularly reasoning and decision-making to the problem of automated software agents has a rich history within the automated planning community. Of particular importance is the work on building softbots for UNIX (Etzioni, Lesh, and Segal 1993; Etzioni and Weld 1994), which is the most comprehensive previous study on this problem. Indeed, as we introduced earlier, many of the issues that are cataloged in that work remain of importance today. The current work builds upon the work of Etzioni et al., particularly their focus on a goal-oriented approach to the problem (Etzioni et al. 1993); however, it goes beyond that work (and related approaches (Petrick 2007)) in actually realizing a learning-based agent framework for the Ubuntu technical support domain. Succinctly, we seek to automate in the largest possible degree the promise of the softbot approach by: (1) exploiting the Bash shell as a robust simulator for learning agents to explore the world; and (2) using the large amounts of data generated by human experts on the internet. On the learning side, Branavan et al.’s work on using reinforcement learning (RL) to map natural language instructions to sequences of executable actions (Branavan et al. 2009) explores a similar problem setting in a Windows OS domain. However, that work focuses on the application of RL techniques to the language processing problem, and on mapping text instructions to executable actions. In contrast, our work focuses on learning task-oriented models for solving the e2eGCS problem. Thus the most relevant prior explorations into this area are complementary to our work in different ways; while the softbot work lays the groundwork for our architecture, Branavan et al.’s work provides a report on using learning on a related but different problem. Contributions The contributions of the paper are • We provide a platform UbuntuWorld 1.0 LTS based on the Ubuntu OS, and its interface to the Bash terminal, where different types of agents can be plugged in and evaluated. This can be adopted as a valuable real-world test bed for research in automated technical support. • We propose data-driven RL as a viable solution to the model learning problem for automated technical support in Ubuntu, by utilizing human intelligence – in the form of data from online technical support forums like Ask Ubuntu – to aid the traditional RL process.